{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pima Indians Diabetes Data Set——Logistic回归\n",
    "数据说明： Pima Indians Diabetes Data Set（皮马印第安人糖尿病数据集） 根据现有的医疗信息预测5年内皮马印第安人糖尿病发作的概率。\n",
    "\n",
    "数据集共9个字段: 0列为pregnants(怀孕次数)； 1列为Plasma_glucose_concentration(口服葡萄糖耐量试验中2小时后的血浆葡萄糖浓度)； 2列为blood_pressure(舒张压,单位:mm Hg） 3列为Triceps_skin_fold_thickness(三头肌皮褶厚度,单位：mm） 4列为serum_insulin(餐后血清胰岛素,单位:mm） 5列为BMI,体重指数（体重（公斤）/ 身高（米）^2） 6列为Diabetes_pedigree_function(糖尿病家系作用) 7列为Age(年龄) 8列为Target(分类变量,0或1）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#导入必要的工具包\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pregnants</th>\n",
       "      <th>Plasma_glucose_concentration</th>\n",
       "      <th>blood_pressure</th>\n",
       "      <th>Triceps_skin_fold_thickness</th>\n",
       "      <th>serum_insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>Diabetes_pedigree_function</th>\n",
       "      <th>Age</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.639947</td>\n",
       "      <td>0.866045</td>\n",
       "      <td>-0.031990</td>\n",
       "      <td>0.670643</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>0.166619</td>\n",
       "      <td>0.468492</td>\n",
       "      <td>1.425995</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.844885</td>\n",
       "      <td>-1.205066</td>\n",
       "      <td>-0.528319</td>\n",
       "      <td>-0.012301</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>-0.852200</td>\n",
       "      <td>-0.365061</td>\n",
       "      <td>-0.190672</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.233880</td>\n",
       "      <td>2.016662</td>\n",
       "      <td>-0.693761</td>\n",
       "      <td>-0.012301</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>-1.332500</td>\n",
       "      <td>0.604397</td>\n",
       "      <td>-0.105584</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.844885</td>\n",
       "      <td>-1.073567</td>\n",
       "      <td>-0.528319</td>\n",
       "      <td>-0.695245</td>\n",
       "      <td>-0.540642</td>\n",
       "      <td>-0.633881</td>\n",
       "      <td>-0.920763</td>\n",
       "      <td>-1.041549</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-1.141852</td>\n",
       "      <td>0.504422</td>\n",
       "      <td>-2.679076</td>\n",
       "      <td>0.670643</td>\n",
       "      <td>0.316566</td>\n",
       "      <td>1.549303</td>\n",
       "      <td>5.484909</td>\n",
       "      <td>-0.020496</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pregnants  Plasma_glucose_concentration  blood_pressure  \\\n",
       "0   0.639947                      0.866045       -0.031990   \n",
       "1  -0.844885                     -1.205066       -0.528319   \n",
       "2   1.233880                      2.016662       -0.693761   \n",
       "3  -0.844885                     -1.073567       -0.528319   \n",
       "4  -1.141852                      0.504422       -2.679076   \n",
       "\n",
       "   Triceps_skin_fold_thickness  serum_insulin       BMI  \\\n",
       "0                     0.670643      -0.181541  0.166619   \n",
       "1                    -0.012301      -0.181541 -0.852200   \n",
       "2                    -0.012301      -0.181541 -1.332500   \n",
       "3                    -0.695245      -0.540642 -0.633881   \n",
       "4                     0.670643       0.316566  1.549303   \n",
       "\n",
       "   Diabetes_pedigree_function       Age  Target  \n",
       "0                    0.468492  1.425995       1  \n",
       "1                   -0.365061 -0.190672       0  \n",
       "2                    0.604397 -0.105584       1  \n",
       "3                   -0.920763 -1.041549       0  \n",
       "4                    5.484909 -0.020496       1  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train = pd.read_csv('FE_pima_indians_diabetes.csv')\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据准备 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = train['Target']\n",
    "X_train = train.drop(['Target'], axis = 1)\n",
    "\n",
    "#保存特征名字以备后用（可视化）\n",
    "feat_names = X_train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 默认参数的Logistics Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "lr = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of each fold is: [0.48797856 0.53011593 0.4562292  0.422546   0.48392885]\n",
      "cv logloss is: 0.47615970944434044\n",
      "accuracy of each fold is: [0.75974026 0.74025974 0.78571429 0.79738562 0.77124183]\n",
      "cv accuracy is: 0.7708683473389355\n"
     ]
    }
   ],
   "source": [
    "#交叉验证用于评估模型性能和进行参数调优（模型选择）\n",
    "#采样5折交叉验证\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "#采用负log似然损失\n",
    "loss1 = cross_val_score(lr, X_train, y_train, cv = 5, scoring = 'neg_log_loss')\n",
    "print('logloss of each fold is:', -loss1)\n",
    "print('cv logloss is:', -loss1.mean())\n",
    "\n",
    "#采用正确率\n",
    "loss2 = cross_val_score(lr, X_train, y_train, cv = 5, scoring = 'accuracy')\n",
    "print('accuracy of each fold is:', loss2)\n",
    "print('cv accuracy is:', loss2.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic Regression + GridSearchCV\n",
    "logistic回归的需要调整超参数有：C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和正则函数penalty（L2/L1） 目标函数为：J = C* sum(logloss(f(xi), yi)) + penalty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False),\n",
       "       fit_params=None, iid=True, n_jobs=4,\n",
       "       param_grid={'penalty': ['l1', 'l2'], 'C': [0.1, 1, 10, 100, 1000]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring='neg_log_loss', verbose=0)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#导入模块\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "#设置参数搜索范围\n",
    "penaltys = ['l1', 'l2']\n",
    "Cs = [0.1, 1, 10, 100, 1000]\n",
    "tuned_parameters = dict(penalty = penaltys, C = Cs)\n",
    "\n",
    "#生成学习器实例\n",
    "lr_penalty = LogisticRegression(solver = 'liblinear')\n",
    "\n",
    "#生成GridSearchCV的实例\n",
    "grid = GridSearchCV(lr_penalty, tuned_parameters, cv = 5, scoring = 'neg_log_loss', n_jobs = 4)\n",
    "\n",
    "#调用GridSearchCV的fit方法\n",
    "grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.4760285431429656\n",
      "{'C': 1, 'penalty': 'l1'}\n"
     ]
    }
   ],
   "source": [
    "#查看最佳模型参数\n",
    "print(-grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "d:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "d:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xt8VPWd//HXZ2aSACHIJQHUQEHFKlKuEW2L1bW6ovUnanV1LVrdtmq3rHbb7Ur3Z22Lbi/a3/rY3fL7VVZFbbVesJZoEbbVequViy0qF61IcY0XCKEgt1wm8/n9cc6ESZjkTCAnCfB+Ph7zmHPOfM85nzmSeXtu32PujoiISEcSPV2AiIj0fgoLERGJpLAQEZFICgsREYmksBARkUgKCxERiaSwEBGRSAoLERGJpLAQEZFIqZ4uoKuUl5f7qFGjeroMEZEDyssvv7zZ3Sui2h00YTFq1ChWrFjR02WIiBxQzOztQtrpMJSIiERSWIiISCSFhYiIRDpozlmIiGQ1NTVRU1NDfX19T5fSa/Tp04fKykqKior2aX6FhYgcdGpqaigrK2PUqFGYWU+X0+Pcnbq6Ompqahg9evQ+LUOHoUTkoFNfX8+QIUMUFCEzY8iQIfu1p6WwEJGDkoKitf3dHgoLERHgkjt+zyV3/L6ny+i1FBYi0qucNP+znDT/sz1dxn7r379/y/D06dMZOHAg5557bsu0Cy64gIkTJ3LMMcdw2GGHMXHiRCZOnMiLL77YqfXMf+ynPLj4kS6ruz06wS0Ss+wP39KrHu3hSqSnfOMb32DXrl3ccccdLdMee+wxAJ555hl+9KMf8cQTT+zTspc+/3sGDR7EpdMv7pJa26M9CxGRmH3605+mrKys4PbLly/n1FNPZcqUKZx99tls3LgRgNtvv52xY8cyYcIEZs6cyVtvvcWCnz3C3XPv2qe9ks7QnoV02lWLrwJg/vT5PVyJSLTvPr6aNe99GNluzftBm0LOW4w9YgDf/l8n7Hdt+TQ0NHD99ddTXV1NeXk5999/P9/61reYN28et956K2+//TbFxcVs3bqVgQMHctHMixk0eBDfv/FfY6knK9awMLPpwL8DSeBOd/9Bm8+vBG4D3g0n/djd7ww/WwycDLzg7ucivUb2j0pEut7atWtZvXo1Z5xxBgDNzc1UVlYCcMIJJzBz5kxmzJjB+eef3611xRYWZpYE5gJnAjXAcjOrdvc1bZo+5O6z8iziNqAfcE1cNYrIwa/QPYDsHsVD13w8znIiuTvjx4/n+eef3+uzJUuW8Oyzz7Jw4UJuueUWVq1a1W11xXnOYiqwzt3Xu3sj8CAwo9CZ3f0pYHtcxYmI9EZjx47l3XffZdmyZQA0NjayevVqmpubqamp4fTTT+e2226jtraWXbt2Udq/lJ07d8ZeV5xhcSTwTs54TTitrc+a2atmtsDMRsRYj4hIjzjllFO4+OKLeeqpp6isrGTJkiXtti0pKWHBggV87WtfY8KECUyaNImlS5eSTqe57LLLGD9+PJMnT+aGG26grKyM088+g8ULn2TSpEkH7AnufLcLepvxx4Gfu3uDmV0L3AucXvAKzK4GrgYYOXLkvtYpItLlduzY0TKc75BS1mmnncZpp53WatrkyZN54YUX9mr7u9/9bq9pR405moXP/Yrjy4/Z92ILEGdY1AC5ewqVwHu5Ddy9Lmf0v4AfdmYF7j4PmAdQVVXVNohERArW0+cqers4D0MtB8aY2WgzKwYuBapzG5jZ4Tmj5wFrY6ynXbrNX0SkY7HtWbh72sxmAUsILp29291Xm9kcYIW7VwPXmdl5QBrYAlyZnd/MngeOA/qbWQ3wBXdv/0CfiIjEJtb7LNx9EbCozbSbcoa/CXyznXlPibM2EREpnO7gBjYU/ygcUt89IiL5qG8oERGA+Z8JXpKXwkJEJAbd0UX53LlzeXzBwi6tuz06DCUiErP96aI8nU6TSuX/qf7KV77C2s3rur7gPLRnISISs852UV5ZWcnNN9/MJz/5SR577DF+8pOfcOKJJzJhwgQuvvhidu/eDcCNN97IfT8Jen+eNm0as2fPZurUqXz0ox/t8ru5tWchIge3J2fDB69Ft/vg1eC9kPMWwz8GZ/8gut1+KC0tbblju66ujmuvvRaA2bNnc8899/DlL395r3ncnWXLllFdXc2cOXNYvHhxl9WjsBAR6YUuueSSluFXX32Vm266ia1bt7J9+/ZW5z5yXXjhhQBMmTKFDRs2dGk9CgsRObgVugeQ3aO46lfx1dIJpaWlLcNXXHEFTz75JOPGjePOO+/kpZdeyjtPSUkJAMlkknQ63aX16JyFiEgvt3PnToYPH05TUxMPPPBAj9SgPQsRkZidcsopvP766+zYsYPKykruuusuzjrrrILnnzNnDlOnTmXkyJGMGzeO+vr6GKvNT2EhIhKD/emivKamptX4rFmzmDVr7weK3nLLLS2XzuZ2aT58+HDWrevaS2oVFiIi0GvOVfRWOmchIiKRFBYiIhJJYSEiIpEUFiIiEklhISICXLX4Kq5afFVPl9FrKSxERGKQ7aJ85cqVfPzjH+eEE05g/PjxPPTQQ0DXdFEO8NLzv+eVFX/s8vrb0qWzIiIx6tevH/fddx9jxozhvffeY8qUKZx11lkFd1EeZenzv2fQ4EFcOv3irix7L7HuWZjZdDN7w8zWmdnsPJ9faWa1ZrYyfH0x57PPm9mb4evzcdYpIhKXY489ljFjxgBwxBFHMHToUGprazucZ/ny5Zx66qlMmTKFs88+m40bNwJw++23M3bsWCZMmMDMmTN56623WPCzR7h77l37tFfSGbHtWZhZEpgLnAnUAMvNrNrd17Rp+pC7z2oz72Dg20AV4MDL4bx/iateETk4/XDZD3l9y+uR7bJtCjlvcdzg47hh6g2drmXZsmU0NjZy9NFHt9umoaGB66+/nurqasrLy7n//vv51re+xbx587j11lt5++23KS4uZuvWrQwcOJCLZl7MoMGD+P6N/9rpejojzsNQU4F17r4ewMweBGYAbcMin7OAX7v7lnDeXwPTgZ/HVKuISKzef/99Lr/8cu69914SifYP6qxdu5bVq1dzxhlnANDc3ExlZSUAJ5xwAjNnzmTGjBmcf/753VJ3VpxhcSTwTs54DXBSnnafNbNPAX8C/tHd32ln3iPbzmhmVwNXA4wcObKLyhaRg0mhewDZPYr50+d3eQ0ffvghn/nMZ7jllls4+eSTO2zr7owfPz5vf1JLlizh2WefZeHChdxyyy2sWrWqy2ttT5znLCzPNG8z/jgwyt3HA78B7u3EvLj7PHevcveqioqK/SpWRCQOjY2NXHDBBVxxxRVcfHH0SeixY8fy7rvvsmzZspb5V69eTXNzMzU1NZx++uncdttt1NbWsmvXLkr7l7Jz5864v0asYVEDjMgZrwTey23g7nXu3hCO/hcwpdB5RUQOBA8//DDPPfcc99xzT8vlsStXrmy3fUlJCQsWLOBrX/saEyZMYNKkSSxdupR0Os1ll13G+PHjmTx5MjfccANlZWWcfvYZLF74JJMmTTowT3ADy4ExZjYaeBe4FLgst4GZHe7u74ej5wFrw+ElwPfMbFA4/tfAN2OsVUSkS2W7KJ85cyYzZ85st12+LsonT57cqsvxrOwzuXMdNeZoFj73K44vP2b/Co4QW1i4e9rMZhH88CeBu919tZnNAVa4ezVwnZmdB6SBLcCV4bxbzOxmgsABmJM92S0iEoc4zlUcTGK9Kc/dFwGL2ky7KWf4m7Szx+DudwN3x1mfiIgURt19iIhIJIWFiIhEUliIiEgkhYWICPD25Vfw9uVX9HQZvZbCQkQkBt3RRfncuXN5fMHCWOpvS12Ui4jEaH+7KE+n06RS+X+qv/KVr7B287rYas+lsBARidGxxx7bMpzbRfnAgQPbnaeyspJrrrmGxYsX89WvfpW6ujruuusuGhsbOfbYY7nvvvvo27cvN954I819nCuuvYpp06Yxbdo0nn76abZt28b8+fP5xCc+0WXfQ2EhIge1D773PRrWRndRXv960KaQ8xYlxx/H8H/5l07XUkgX5VmlpaUtd2zX1dVx7bXXAjB79mzuuecevvzlL+81j7uzbNkyqqurmTNnDosXL+50je1RWIiIdINCuyjPuuSSS1qGX331VW666Sa2bt3K9u3bOffcc/POc+GFFwIwZcoUNmzY0CV1ZyksROSgVugeQHaP4iM/va/La+hMF+VZpaWlLcNXXHEFTz75JOPGjePOO+/kpZdeyjtPSUkJAMlkknQ6vf+F59DVUCIiMepsF+X57Ny5k+HDh9PU1MQDDzzQxRUWRnsWIiIxynZRXldXxz333APQ0l15oebMmcPUqVMZOXIk48aNo76+PqZq26ewEBGJwf50UV5TU9NqfNasWcyaNWuveW+55ZaWS2dzuzQfPnw469Z17SW1CgsREeI5V3Ew0TkLERGJpLAQkYOSu/d0Cb3K/m4PhYWIHHT69OlDXV2dAiPk7tTV1dGnT599XobOWYjIQaeyspKamhpqa2t7upTYvb8j/I61TR2269OnD5WVlfu8nljDwsymA/9O8AzuO939B+20uwh4BDjR3VeYWTFwB1AFZIDr3f2ZOGsVkYNHUVERo0eP7ukyusWV828EYOlVj8a6ntjCwsySwFzgTKAGWG5m1e6+pk27MuA6YGnO5C8BuPvHzGwo8KSZnejumbjqFRGR9sV5zmIqsM7d17t7I/AgMCNPu5uBW4Hcu0zGAk8BuPsmYCvBXoaIiPSAOMPiSOCdnPGacFoLM5sEjHD3th25vwLMMLOUmY0GpgAj2q7AzK42sxVmtuJQODYpItJT4jxnYXmmtVyaYGYJ4Hbgyjzt7gaOB1YAbwMvAnv1iuXu84B5AFVVVbrsQUQkJnGGRQ2t9wYqgfdyxsuAccAzZgYwHKg2s/PcfQXwj9mGZvYi8GaMtYqISAfiPAy1HBhjZqPDq5suBaqzH7r7Nncvd/dR7j4KeAk4L7waqp+ZlQKY2ZlAuu2JcRER6T6x7Vm4e9rMZgFLCC6dvdvdV5vZHGCFu1d3MPtQYImZZYB3gcvjqlNERKLFep+Fuy8CFrWZdlM7bU/LGd4AfDTO2kREpHDq7kNERCIpLEREJJLCQkREIiksREQkksJCREQiKSxERCSSwkJERCIpLEREJJLCQkREIiksREQkksJCREQiKSxERCSSwkJERCIpLEREJJLCQkREIiksREQkUqfDwswSZjYgjmJERKR3KigszOwBMxsQPhd7DfCGmX0j3tJERKS3KHTPYqy7fwicT/CY1JEU8FxsM5tuZm+Y2Tozm91Bu4vMzM2sKhwvMrN7zew1M1trZt8ssE4REYlBoWFRZGZFBGGx0N2bAO9oBjNLAnOBs4GxwN+a2dg87cqA64ClOZMvBkrc/WPAFOAaMxtVYK0iItLFCg2LO4ANQCnwnJl9BPgwYp6pwDp3X+/ujcCDwIw87W4GbgXqc6Y5UGpmKaAv0FjA+kREJCYFhYW7/4e7H+nu53jgbeCvImY7EngnZ7wmnNbCzCYBI9z9iTbzLgB2Au8D/wP8yN23FFKriIh0vUJPcF8fnuA2M7vLzP4AnB41W55pLYeuzCwB3A58PU+7qUAzcAQwGvi6mR2Vp66rzWyFma2ora0t5KuIiMg+KPQw1N+FJ7j/GqgArgJ+EDFPDTAiZ7wSeC9nvAwYBzxjZhuAk4Hq8CT3ZcBid29y903A74Cqtitw93nuXuXuVRUVFQV+FRER6axCwyK7l3AOMN/dXyH/nkOu5cAYMxttZsXApUB19kN33+bu5e4+yt1HAS8B57n7CoJDT6eHezKlBEHyesHfqhMyGaepfgjNTaU0pjNxrEJE5ICXKrDdy2b23wSHhL4ZXsHU4S+ru6fNbBawBEgCd7v7ajObA6xw9+oOZp8LzAdWEYTSfHd/tcBaO+UvuxrZ8nZw3v3YG5/ksL5FlPcvprx/CeVlJVT0L6G8fzEVZSXBtHB6ef9iSlLJOEoSEel1Cg2LLwATgfXuvsvMhhAciuqQuy8iuC8jd9pN7bQ9LWd4B8Hls7ErLUlx2BG/IdPcly+Nv4rNOxqo3d7A5h0NrHnvQzZvb2B7QzrvvGV9Ui0hkg2V8v4le4KlbM+0PkUKFhE5cBUUFu6eMbNK4DIzA3jW3R+PtbJu0qcoyUf7/BaA6z79vbxt6puaWwJk847G4D0cr93RwObtjax9/0NqdzSwvb6dYClJ5YRI8Z69lJZw2RM0ChYR6W0KCgsz+wFwInB/OOk6M/uEux8Sd1b3KUoyYnA/RgzuF9m2vql5T6i0BEwwXrs9CJfXP9jO5u2b+bCdYOnfEiytQ6W8rDjYg2k5PFZC32IFi4jEr9DDUOcAE909A2Bm9wJ/BA74sKhP17MlmcEcfvHmLyhKFJFKpEglUi3Dbd/b+yw7fMTAPlQOig6WhnQzdWGI5AuVzdsb+NPG7bz4Vh3bdjflXUZpcbJVeOTutbQ6RFZWTL/iQv9zi4i01plfj4FA9sa4w2KopUfsaNrBxlRw+8e3X/x2ly03acmCQ2avzwYU0X9gikGJIo4PP0taEU1paEwnaGiChiZjdyPsbnR21cP2Buf9bbB9Y4ZdDeCehJZXAvcUfVJFDOrXl8H9+jK4Xx+GlPZjSGlfKspKGdq/L8MHlDKsrJShZX0pLemeYHF3HKfZm3EP3jOeafXq6LPcNhnPkCFDJhO+t2mTbm6mKdNMOtNMU3OadCZDU6aZ5vA93Rx8FrwyLe/NmXT4HnzW7Jlg3Jtpzo4376khO605XO/O+gwYXLLgn1t/dxz31lNaD2U/91afenbI24znzultp5OzrtbLa9XSO/gsZ9lte/vJXXbratq0w3Nm3XvZQMuh3NN/+gW6ROuNnGeNcYt3TTvqmzDvE+s6oPCw+D7wRzP7LcHVSZ/iINirABjcZzBjGoIriOdetph0Jk1TpommTFPLcEfvhXyWO729+Xand/Nh5sOCltnszXt/kaLwVRb0j9KeD8PXBgd2hK+Nrdu4G3gSI0nCUiQtRcrCIEum2OVbATjjoXPJeAYn+GF0dzLeTIZwmOYwDDJ4+EPungl+JAnm684/2bi5G2DgRnBVejhcYuAJVm/bc5tR62+d/yp06+Czjq9ct3aGOxordHm5U9pO3/d2wdSwdSoIiy2Nuzqo6UATdafBfkg1kUnH/2iiQk9w/9zMniE4b2HADe7+QZyFdZeEJUiF/yEP7394D1dTmIxn2g2UDsOsuYm073lPZ9LUpxv5sL6erbt3s62+ge319exoaGRHYwO7mhrY2djI7qYG6tNNNDY3gTVD+DiTmp3ZH8ScH0YS+F7Twh9PNxKWwCxBgvA9HE5khy1JwoykJbHwPdHyniCZSJK0BIlEgqQlSFqyZVrSEqSSQbtUOC2VSJJMJEiF7VLJYFoqkX1v/SpKBp8VJYL2xck976lEgqJkkqJkEUXJoE0wnqQ4kSKVSpBKGMmE5bwnOPWnl4I5L37+YcILRFp+OsxomSaBk+Z/FoClVz3aw5UcGE6a/9nOHSPaRx2uwswmt5lUE74fYWZHuPsf4ilLOpKwBMXJYoqTxd263nRzhi07G5l+/z8Axv3n/6jVj2IyaXl/LLPjicSh+aNoieCWpFRSD6aUA1dUHv2fDj5zovuHkoNIKplg6IA+FPUJTl0df7gemChyqOgwLNw9qmdZERE5BBR6n8WFeSZvA14LO/oTEZGDWGe6+/g48Ntw/DSCjv+ONbM57v7TGGoTEZFeotCwyADHu/tGADMbBvw/4CTgOUBhISJyECs0LEZlgyK0CTjW3beYWf5biw8gVz7cGAxc3bN1iIj0VoWGxfNm9gTwSDh+EcGzuEuBrbFUJiIivUahYfEV4EJgGsH9RPcCj7q7E/0sbhEROcAVege3m9kLQCPB/RXLwqAQEZFDQEG3lJrZ3wDLCA4//Q2w1MwuirMwERHpPQo9DPW/gROz91SYWQXwG2BBXIWJiEjvUWhnNYk2N9/VFTKvmU03szfMbJ2Zze6g3UVm5mZWFY5/zsxW5rwyZjaxwFpFRKSLFbpnsdjMlgA/D8cvoc2ztdsysyQwFziToAPC5WZW7e5r2rQrA64Dlmanufv9hE/lM7OPAQvdfWWBtUrMvn3/2mAg8insAtpenaXt1Tndtb0K2rNw928A84DxwARgnrvfEDHbVGCdu69390bgQWBGnnY3A7cC9e0s52/ZE1IiItIDCu4F3d0fBTrTwfyRwDs54zUEd3y3MLNJwAh3f8LM/qmd5VxC/pDpGplmxo/5C3/ZXgyb34QhxwQPGRARkRZRz7PYTv5HmRnBFbUd9VGd7xe3ZVlmlgBuB67sYP0nAbvcfVU7n19NeN/1yJEjOyilAx++R/nAeiqH7YIfV0H/YTBqGow6JXgNOVrhISKHvKguysv2Y9k1wIic8UrgvZzxMmAc8Ez4pLDhQLWZnefuK8I2l9LBISh3n0dweIyqqqp9u+9j4AieXj6cfn2aOe27/wIbXoA/Pw+rwp2o/sPD8JgGoz8Fg49SeIjIISfOh/EtB8aY2WjgXYIf/suyH7r7NqA8Ox4+tvWfskER7nlcTPC875gZu+pTMOXK4OUOW9bDn58LwmPDC7AqvEq47PA94THqFIWHiBwSYgsLd0+b2SxgCZAE7nb31WY2B1jh7tURi/gUUOPu6+OqsV1mweGnIUdD1VVBeNS9BRueD15/fg5eC7vJKjsiJzymKTxE5KAU62O+3X0RbS6xdfeb2ml7WpvxZ4CT46qtU8yg/Jjg1RIe68LweAHWPwOvPRy0HXBk6/AYNFrhISIHvFjD4qBlBuVjglfV3wXhsfnNPeHx1m/h1YeCti3hcUoYHqMUHiJywFFYdAUzqDg2eJ34hTA8/pQTHk/nhEdlzgnzU2DgRxQeItLrKSziYAYVHw1eJ35xT3hkT5iv+w28+mDQ9rARrU+YD/pIz9YuIpKHwqI75IbH1C8F4VH7xp4T5m/+N7wSXiF82Mg25zwUHiLS8xQWPcEMhh4XvFrC4/XwMt3n4c0l8MoDQduBI/ec7xg1LRgXEelmh3xYuDuptJNO9mARZjD0+OA19UuQybQOjzeehJX3B21bwiMMkIEjOl62iEgXOOTDonnrVkZ8EAz/6eSPkxo2jNTQoaSGVpAaOpSi7HjFUFLDhpIaMgRLxpwsiQQMGxu8Tro6DI+17YTHR4LgGB2Gx2GV8dYmIoekQz4sEsXF1A6CVDOMnn4W6U21pDdtouGNN0hv3hz8ULeaIUGqvDwIkGHDSA2toGjoUFJDsyEzlKJhQ0kcdhjWVVc5JRIw7ITgddI1e8Ljz+E5jzd+BSt/FrQdNKr1pboKDxHpAgqL0lJ2lAY/6od/5zutPvN0mnTdFtKbNpHetJH0pk00bdpEeuOmYPidd9j98ss0b92613KtuLglPFLDhoaBkhsqFRQNG0aiX799KDonPE6+NgiPTWv27HmsfQL+mA2P0W3C48jOr09EDnmHfFh0xFIpioYFewpBn4f5ZRoaSNcGeyTpjXuHSsPa19nx7HP4rl17zZvo33/vQKlos9dSUYEVF7dfaCIBw8cFr5bwWL2nX6u1j8Mffxq0HXxU6/AYcMR+biURORQoLLpAoqSE4spKiivbP+Tj7mR27gz3UoJQadq0KTjsFQbMruUraKqthaamveZPDh7cOlRyAiU1NJiWHDIESyTC8PhY8Dr5y0F4bFy1JzzWLIQ/3BcsuCU8PgWjPqnwEJG8FBbdxMxI9u9Psn9/So46qt12nsnQvHVrS6g0hUGSGyr1a9bQvLkuuOQ2VyrVcj6laFgYKK3OrZxJauznSPQvxXL3PFbnhsfRe7pj/8gnYcDhMW4VETlQKCx6GUskSA0eTGrwYDjuuHbbeVMT6bq6NqGy51BY44YN7Fy2nMy2bXuvo0+fPedNhh5BquLvSQ1qJpXZRNHOt0j9fiGpZfeRSHnw5MDcw1Zlw+P8+iLSSyksDlBWVETR8OEUDe/4xzuze/ee8ymbNtEUnkfJhsru1atIb9yE1+c+Ar0f0I9EvxJSpc0UpRaRKvklqb7NpMqHcNyAHTS5seM/ZwXPQzQDDBLhu+39brnTWl7hI+CzV41ZzjyJBJBnvpbPaL2e7LJyPrPsZ+Gy2GtZibBm9owbkEiGk7LzESy/ZT0537Wl7sSedWPBvOG6D0s04QYNy39T0H/bQ92AZHAYtmHF0z1cyYFhQLKJjMffv5zC4iCX6NuX4pEjKe7gsbPuTmb79j2BknNyPrvX0vDBu6Tf+QusbQRKAXhn7VPd9C0ObIPDP7P1l/9DD1dyYBiS3V4zv9LDlRwYhpCiaODe5zm7msJCgvMpAwaQHDCAkmOOabedZzI0b9nCMxeegjl84j9/Hp43cTyTCYY9Q8uj1jPhcPYdh0zuO7g3h82z84bzuQOZPedlsstwb5nm2XVll+lt27NnWnbZOB6ue0973/Oeb1nhvJ7Jadt2vnzTwvZ/fjQ4HzT6wssL/m9yKPvzL4Ir90Zd8LkeruTAsOGx+9nVDR1XKyykYBbekNhYHPzL7DtxYg9XdGB49+Gge/oJ13ynZws5QLz7SPAgsQnXzunhSg4M7y5Y0C3rSUQ3ERGRQ12sYWFm083sDTNbZ2azO2h3kZm5mVXlTBtvZr83s9Vm9pqZ9YmrzgbrS4P1jWvxIiIHvNgOQ5lZEpgLnAnUAMvNrNrd17RpVwZcByzNmZYCfgZc7u6vmNkQIP4zOCIiklecexZTgXXuvt7dG4EHgRl52t0M3ArkXrv518Cr7v4KgLvXuXtzjLWKiEgH4gyLI4F3csZrwmktzGwSMMLdn2gz77GAm9kSM/uDmf1zjHWKiEiEOK+GyncxV0v/FBbc8XQ7cGWedilgGnAisAt4ysxedvdWF/ab2dXA1QAjO7iPQERE9k+cexY1QO5j3CqB93LGywi6cn3GzDYAJwPV4UnuGuBZd9/s7ruARcDktitw93nuXuXuVRUVFTF9DRERiTMslgNjzGy0mRUDlwLV2Q/dfZu7l7v7KHcfBbwEnOfuK4AlwHhxXrM4AAALBklEQVQz6xee7D4VWLP3KkREpDvEFhbungZmEfzwrwUedvfVZjbHzM6LmPcvwL8RBM5K4A/u/qu4ahURkY7Fege3uy8iOISUO+2mdtqe1mb8ZwSXz4qISA/THdwiIhJJfUMBP7hsFAAX9GwZIiK9lvYsREQkksJCREQiKSxERCSSwkJERCIpLEREJJLCQkREIiksREQkksJCREQi6aY86bTvfu54AM7p4ToOFNpenaPt1Tndtb20ZyEiIpEUFiIiEklhISIikRQWIiISSWEhIiKRFBYiIhJJYSEiIpEUFiIiEinWsDCz6Wb2hpmtM7PZHbS7yMzczKrC8VFmttvMVoavn8RZp4iIdCy2O7jNLAnMBc4EaoDlZlbt7mvatCsDrgOWtlnEW+4+Ma76RESkcHHuWUwF1rn7endvBB4EZuRpdzNwK1AfYy0iIrIf4gyLI4F3csZrwmktzGwSMMLdn8gz/2gz+6OZPWtmp+RbgZldbWYrzGxFbW1tlxUuIiKtxRkWlmeat3xolgBuB76ep937wEh3nwR8DXjAzAbstTD3ee5e5e5VFRUVXVS2iIi0FWdY1AAjcsYrgfdyxsuAccAzZrYBOBmoNrMqd29w9zoAd38ZeAs4NsZaRUSkA3GGxXJgjJmNNrNi4FKgOvuhu29z93J3H+Xuo4CXgPPcfYWZVYQnyDGzo4AxwPoYaxURkQ7EdjWUu6fNbBawBEgCd7v7ajObA6xw9+oOZv8UMMfM0kAzcK27b4mrVhER6VisDz9y90XAojbTbmqn7Wk5w48Cj8ZZm4iIFE53cIuISCQ9VhUYe/heF1qJiEgO7VmIiEgkhYWIiERSWIiISCSFhYiIRFJYiIhIJF0NBcyfPr+nSxAR6dW0ZyEiIpEUFiIiEklhISIikRQWIiISSWEhIiKRFBYiIhJJYSEiIpEUFiIiEklhISIikRQWIiISKdawMLPpZvaGma0zs9kdtLvIzNzMqtpMH2lmO8zsn+KsU0REOhZbWJhZEpgLnA2MBf7WzMbmaVcGXAcszbOY24En46pRREQKE2dHglOBde6+HsDMHgRmAGvatLsZuBVotfdgZucD64GdMdYo+2DpVY/2dAki0s3iDIsjgXdyxmuAk3IbmNkkYIS7P5F7qMnMSoEbgDNpEyJt5r8auBpg5MiRXVe5SBdSuHaOtlfndNf2ivOcheWZ5i0fmiUIDjN9PU+77wK3u/uOjlbg7vPcvcrdqyoqKvarWBERaV+cexY1wIic8UrgvZzxMmAc8IyZAQwHqs3sPII9kIvM7FZgIJAxs3p3/3GM9YqISDviDIvlwBgzGw28C1wKXJb90N23AeXZcTN7Bvgnd18BnJIz/TvADgWFiEjPie0wlLungVnAEmAt8LC7rzazOeHeg4iIHCDM3aNbHQCqqqp8xYoVPV2GiMgBxcxedveqqHa6g1tERCIpLEREJJLCQkREIiksREQk0kFzgtvMaoG392MR5cDmLiqnK6muzlFdnaO6OudgrOsj7h55V/NBExb7y8xWFHJFQHdTXZ2jujpHdXXOoVyXDkOJiEgkhYWIiERSWOwxr6cLaIfq6hzV1Tmqq3MO2bp0zkJERCJpz0JERCIdsmFhZheb2Wozy7R99nebdgU9R7wL6xpsZr82szfD90HttGs2s5XhqzrGejr8/mZWYmYPhZ8vNbNRcdXSiZquNLPanO3zxbhrCtd7t5ltMrNV7XxuZvYfYd2vmtnkXlLXaWa2LWd73dRNdY0ws9+a2drwb/H6PG26fZsVWFe3bzMz62Nmy8zslbCu7+ZpE9/fo7sfki/geOCjwDNAVTttksBbwFFAMfAKMDbmum4FZofDs4EfttNuRzdso8jvD/w98JNw+FLgoV5Q05XAj3vg39SngMnAqnY+P4fgmfIGnAws7SV1nQY80QPb63BgcjhcBvwpz3/Lbt9mBdbV7dss3Ab9w+EiYClwcps2sf09HrJ7Fu6+1t3fiGjW8hxxd28Ess8Rj9MM4N5w+F7g/JjX15FCvn9uvQuAT1v4NKserKlHuPtzwJYOmswA7vPAS8BAMzu8F9TVI9z9fXf/Qzi8neBRBke2adbt26zAurpduA2yTw8tCl9tTzrH9vd4yIZFgfI9RzzufzTD3P19CP7RAkPbadfHzFaY2UtmFlegFPL9W9p48AyTbcCQmOoptCaAz4aHLRaY2Yg8n/eEnvj3VKiPh4c3njSzE7p75eHhkkkE/7ecq0e3WQd1QQ9sMzNLmtlKYBPwa3dvd3t19d9jnE/K63Fm9huCx7W29b/dfWEhi8gzbb8vH+uork4sZqS7v2dmRwFPm9lr7v7W/tbWRiHfP5Zt1IFC1vc48HN3bzCzawn+T+v0GGsqVHdvq0L9gaDLhx1mdg7wS2BMd63czPoDjwJfdfcP236cZ5Zu2WYRdfXINnP3ZmCimQ0EHjOzce6eey4qtu11UIeFu5+xn4uIeo74PumoLjPbaGaHu/v74e72pnaW8V74vt6CR9JOIjiW35UK+f7ZNjVmlgIOI95DHpE1uXtdzuh/AT+MsZ7OiOXf0/7K/SF090Vm9n/NrNzdY+8DycyKCH6Q73f3X+Rp0iPbLKquntxm4Tq3hn/304HcsIjt71GHoTrW8hxxMysmOGEU25VHoWrg8+Hw54G99oDMbJCZlYTD5cAngTUx1FLI98+t9yLgaQ/PrsUksqY2x7TPIzjm3BtUA1eEV/icDGzLHnLsSWY2PHtc28ymEvwu1HU8V5es14C7gLXu/m/tNOv2bVZIXT2xzcysItyjwMz6AmcAr7dpFt/fY3eeze9NL+ACghRuADYCS8LpRwCLctqdQ3A1xFsEh6/irmsI8BTwZvg+OJxeBdwZDn8CeI3gSqDXgC/EWM9e3x+YA5wXDvcBHgHWAcuAo7phG0XV9H1gdbh9fgsc103/pn4OvA80hf+2vgBcC1wbfm7A3LDu12jnKrweqGtWzvZ6CfhEN9U1jeAQyavAyvB1Tk9vswLr6vZtBowH/hjWtQq4KZzeLX+PuoNbREQi6TCUiIhEUliIiEgkhYWIiERSWIiISCSFhYiIRFJYiHSCme2IbtXh/AvCu+4xs/5mdoeZvRX2IvqcmZ1kZsXh8EF906wcWBQWIt0k7D8o6e7rw0l3EtxdO8bdTyDoLbfcgw4SnwIu6ZFCRfJQWIjsg/CO4tvMbJWZvWZml4TTE2HXD6vN7AkzW2RmF4WzfY7wjnwzOxo4CbjR3TMQdN3i7r8K2/4ybC/SK2g3V2TfXAhMBCYA5cByM3uOoOuVUcDHCHoMXgvcHc7zSYK7qQFOAFZ60DFcPquAE2OpXGQfaM9CZN9MI+jZttndNwLPEvy4TwMecfeMu39A0N1I1uFAbSELD0Ok0czKurhukX2isBDZN+09UKajB83sJui7B4J+hSaYWUd/gyVA/T7UJtLlFBYi++Y54JLwYTQVBI8uXQa8QPDgpYSZDSN4/GbWWuAYAA+ePbIC+G5O76VjzGxGODwEqHX3pu76QiIdUViI7JvHCHr/fAV4Gvjn8LDTowQ9u64C7iB4wtq2cJ5f0To8vkjwEKx1ZvYawbM3ss9q+CtgUbxfQaRw6nVWpIuZWX8PnqA2hGBv45Pu/kH4DILfhuPtndjOLuMXwDc9+jnxIt1CV0OJdL0nwofUFAM3h3scuPtuM/s2wXOS/6e9mcOHOv1SQSG9ifYsREQkks5ZiIhIJIWFiIhEUliIiEgkhYWIiERSWIiISCSFhYiIRPr/4eGlL2dGXXYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb4985c0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid.cv_results_['mean_test_score']\n",
    "test_stds = grid.cv_results_['std_test_score']\n",
    "train_means = grid.cv_results_['mean_train_score']\n",
    "train_stds = grid.cv_results_['std_train_score']\n",
    "\n",
    "# plot results\n",
    "n_Cs = len(Cs)\n",
    "number_penaltys = len(penaltys)\n",
    "test_scores = np.array(test_means).reshape(n_Cs, number_penaltys)\n",
    "train_scores = np.array(train_means).reshape(n_Cs, number_penaltys)\n",
    "test_stds = np.array(test_stds).reshape(n_Cs, number_penaltys)\n",
    "train_stds = np.array(train_stds).reshape(n_Cs, number_penaltys)\n",
    "\n",
    "x_axis = np.log10(Cs)\n",
    "for i, value in enumerate(penaltys):\n",
    "    plt.errorbar(x_axis, -test_scores[:,i], yerr = test_stds[:,i], label = penaltys[i] + 'Test')\n",
    "    plt.errorbar(x_axis, -train_scores[:,i], yerr = train_stds[:,i], label = penaltys[i] + 'Train')\n",
    "    \n",
    "plt.legend()\n",
    "plt.xlabel('log(C)')\n",
    "plt.ylabel('logloss')\n",
    "plt.savefig('LogisticGridSearchCV_C.png')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上图给出了L1正则和L2正则下、不同正则参数C对应的模型在训练集上和测试集上的logloss。 可以看出在训练集上C越大（正则越少）的模型性能越好；在测试集上当C=1时性能最好。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 保存模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "pickle.dump(grid.best_estimator_, open('Diabetes_L1_org.pkl', 'wb'))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
